Recognizing film and video occurring in parallel in television fields

ABSTRACT

A motion sequence pattern detector ( 300,301 ) for detecting presence of film material in a series of consecutive video fields (pp,p,c), is arranged to compute for a first one of the consecutive fields a value of a video motion measure and a value of a film motion measure and to determine the presence of film material on basis of both motion measures. The value of the video motion measure is computed by: establishing a plurality of motion patterns for respective groups of pixels of the first one of the consecutive fields; comparing each of the plurality of motion patterns with a predetermined video motion pattern and conditionally increasing the value of the video motion measure. The value of the film motion measure is computed by comparing each of the plurality of motion patterns with a predetermined film motion pattern and conditionally increasing the value of the film motion measure.

The invention relates to a motion sequence pattern detector fordetecting presence of film material in a series of consecutive videofields.

The invention further relates to an image processing apparatus,comprising:

receiving means for receiving a signal corresponding to a series ofconsecutive video fields;

such a motion sequence pattern detector; and

an image processing unit for computing a sequence of output images onbasis of the series of consecutive video fields, the image processingunit being controlled by the motion sequence pattern detector.

The invention further relates to a method of detecting presence of filmmaterial in a series of consecutive video fields.

The invention further relates to a computer program product to be loadedby a computer arrangement, comprising instructions to detect presence offilm material in a series of consecutive video fields.

When focussing on picture rates, three formats can be distinguished:

50 Hz video: A transmission standard, commonly known as PAL or SECAMthat comprises 50 interlaced fields per second. Each frame comprises 625lines of which the even and odd lines are alternatingly transmitted asfields. The 50 Hz video standard is used in most countries throughoutthe world except Japan and North America.

60 Hz video: A transmission standard, commonly known as NTSC thatcomprises 60 (59.94 to be exact) interlaced fields per second. Eachframe comprises 525 lines of which the even and odd lines arealternatingly transmitted as fields. The 60 Hz video standard is used inJapan and North America.

24 Hz film: Film corresponds to a method of recording moving images on along strip of transparent material. The frame rate of 24 images persecond is a compromise between the ability to capture motion and theamount of film required per time interval. The standard is older thanthe video transmission standards. Attempts were made to adapt the framerate to 25 and 30 images per second, in order to become more compatiblewith transmission standards. Except for some exceptions, e.g.commercials, these frame rates did not find major much support in themotion picture industry. Therefore, 24 Hz film remains the most commonlyused standard for motion pictures.

When television became a popular medium, the need for new contentincreased. This called for format conversion methods. Besides convertingmotion pictures to television, television shows were exchanged betweendifferent transmission standards. This content also needed conversion.Later, when the television was dominant, video material was converted tofilm, e.g. to show television commercials in cinemas. Because of bothartistic and economic reasons, the motion picture industry still appliesthe same procedure to transfer the film format to the video formats.

The process to transfer film to video is called the telecine process.One of the many implementations of this process is to illuminate thefilm and capture light coming through the film with a video camera andadvancing the film in the vertical blanking period of the video signal.To change the frame rate from 24 Hz film to 50 Hz video or 60 Hz video,a process called “pull-down” is used. Pull-down is a method where theprevious picture of the film is repeated until a new one is available.This method can easily be implemented mechanically. To transfer 24 Hzfilm to 50 Hz video, the picture rate of the film is increased to 25pictures per second by running the film slightly faster. The fourpercent increase of speed and pitch of the sound is not regarded asannoying by the general public. Then, each film picture is scannedtwice, creating two video fields. This method is called 2:2 pull-down.See also FIG. 1B. To transfer 24 Hz film to 60 Hz video, speed up to 30Hz is not desired, since the speed up and the change in pitch of thesound is regarded as unacceptable by the general public. Thereforeanother method is used, where every even film picture is repeated threetimes while every odd film picture is repeated two times. This createsan increase of frame rate by a factor 2.5, resulting in a 60 Hz videosignal. This method is called 3:2 pull down. See also FIG. 1C.

An image processing apparatus, like a TV, might comprise an imageprocessing unit for computing from a series of original input images alarger series of output images. In that case, a number of the outputimages are temporally located between successive original input images.This computing is typically known as image rate conversion. For imagerate conversion it is relevant to determine the type of the acquisitionsource of the received images. That means that for achieving a goodimage quality, it has to be detected whether the received imagesoriginate from a film camera which acquired images in a progressive scanmode at a lower image rate or originate from a video camera whichacquired images at the image rate of the video signal. Based on thatdetection, the received video fields are combined to form images. In thecase that the received video fields correspond to film then twosuccessive fields can be merged relatively easily. In the case that thereceived video fields correspond to video then an interpolation ofpixels values of the video fields is required which is controlled by thedetected motion in the images. Incorrect handling of a video mode signalas film mode can cause severe artifacts which are clearly visible in theoutput images. These artifacts are known as “forks”, “mouse teeth”,“comb effect” or “zippers”. False video mode detection is less severe,but also yields artifacts.

In general, the signal as received by the image processing apparatusdoes not comprise an explicit indication of the type of acquisitionsource of the succession of the video fields. As a result, thisinformation has to be extracted from the video fields themselves.Typically this is done by means of detecting a motion sequence pattern.

An embodiment of the motion sequence pattern detector of the kinddescribed in the opening paragraph is known from U.S. Pat. No.4,982,280. This patent specification discloses a motion sequence patterndetector being arranged to detect a periodic pattern of motion sequenceswithin a succession of video fields, such as film mode or progressivescan mode. The motion sequence pattern detector comprises a motiondetector for detecting the presence of motion from increment toincrement within predetermined increments of the succession of videofields and for thereupon outputting a first motion detection signal foreach said increment. The motion detector computes differences betweenpixel values of successive video fields and compares the computationresults with a threshold to reduce the effect of noise. The motionsequence pattern detector further comprises logic circuitry responsiveto the first motion detection signal for detecting the periodic patternof motion sequences within the succession of video fields.

Nowadays it is fashionable to have banners, i.e. scrolling texts, andother information superimposed on video data origination from an othersource. In general, these scrolling texts are in video mode. The videodata upon which they are superimposed, can be in film mode. The resultis a sequence of video fields that contains both objects or regions infilm mode and objects in video mode (See FIG. 5). This kind of sequencesare called hybrid sequences.

Besides this mixing or superimposing, some compression algorithms arearranged to encode parts of the sequence in such a manner, that 2:2pull-down is introduced. An example of such a compression algorithms isDV (Digital Video) coding. In DV coding, parts of the image are encodedon frame basis, while other parts are encoded on field basis. This is toincrease coding efficiency. Coding artifacts may cause motion patternssimilar to hybrid signals.

Most available film detectors are not designed to deal with hybridsequences, since they are arranged to classify sequences as either filmmode or as video mode. E.g. for frame-rate conversion, thisclassification does not suffice. So, such detectors are unreliable onhybrid signals. If a hybrid sequence is detected as film mode, annoyingartifacts are introduced by the frame-rate conversion in the regionsthat are in video mode.

In patent application US2002/0131499 a hybrid detector is disclosed.This detector works as follows. Prior to detecting a film mode, thefields of the television signal are separated into different objects bymeans of a segmentation technique. Any known technique to do so might beused for that purpose. Then, the film mode of each individual object isdetected. Any known film mode detection technique might be used for thatpurpose. In this context, an “object” may be a portion of an individualimage in a field. An “object” is defined as an image portion that can bedescribed with a single motion model. Such an “object” need notnecessarily comprise one “physical” object, like a picture of oneperson. An object may well relate to more than one physical object,e.g., a person sitting on a bike where the movement of the person andthe bike, essentially, can be described with the same motion model. Onthe other hand, one can safely assume that objects identified in thisway belong to one single image originating from one single film source.

A disadvantage of the known hybrid detector is that a separatesegmentation step is required. The more so, since robust segmentation isin general relatively complex.

It is an object of the invention to provide a motion sequence patterndetector of the kind described in the opening paragraph which isarranged to deal with hybrid sequences and which is relatively simple.

This object of the invention is achieved in that the motion sequencepattern detector comprises processing means which is arranged:

to compute for a first one of the consecutive fields a value of a videomotion measure and a value of a film motion measure; and

to determine the presence of film material on basis of the value of thevideo motion measure and the value of the film motion measure, the valueof the video motion measure being computed by:

establishing a plurality of motion patterns for respective groups ofpixels of the first one of the consecutive fields;

comparing each of the plurality of motion patterns with a predeterminedvideo motion pattern and conditionally increasing the value of the videomotion measure, the value of the film motion measure being computed by:

comparing each of the plurality of motion patterns with a predeterminedfilm motion pattern and conditionally increasing the value of the filmmotion measure.

Instead of segmenting the field into objects with semantic meaning, aplurality of groups of pixels are created, e.g. by means ofsub-sampling. The number of these groups is in the order of the numberof pixels in a field, e.g. 10% or 50% of the total number of pixels inthe field. Preferably the groups of pixels each have one pixel only. Foreach of these groups of pixels a motion pattern is established and twopattern matches are performed. The processing means is arranged to checkwhether the established motion pattern corresponds with a typical videopattern or whether the established motion pattern corresponds with atypical film pattern. After these checks, for the corresponding group ofpixels the probable mode, i.e. film mode or video mode, for that groupof pixels is known. By counting for the first one of the consecutivefields the number of times it is decided that a group of pixels has afilm mode the film motion measure for that field is determined. Bycounting for the first one of the consecutive fields the number of timesit is decided that a group of pixels has a video mode, the video motionmeasure for that field is determined. The eventual classification ismade based on the ratio between and values of the video motion measureand the film motion measure:

the value of the film motion measure is relatively high and the value ofthe video motion measure is relatively low. So, the field primarilycomprises material originating from a film camera, i.e. the fieldcorresponds to film mode;

the value of the video motion measure is relatively high and the valueof the film motion measure is relatively low. So, the field primarilycomprises material originating from an interlaced video camera, i.e. thefield corresponds to video mode;

the value of the video motion measure and the value of the film motionare comparable. So, the field comprises material originating from aninterlaced video camera but also material originating from a filmcamera, i.e. the field corresponds to a hybrid mode.

the value of the video motion measure is relatively low and the value ofthe film motion measure is relatively low. No significant motion hasbeen detected, i.e. the field corresponds to a static mode.

In an embodiment of the motion sequence pattern detector according tothe invention the processing means are arranged to establish a first oneof the motion patterns by computing:

a first difference between a first pixel value of the first one of theconsecutive fields and a second value being derived from a second one ofthe consecutive fields; and

a second difference between a third pixel value of a third one of theconsecutive fields and a fourth value being derived from the second oneof the consecutive fields.

Hence, the motion pattern comprises two differences between valuesderived from subsequent fields. The computation of such a pattern isrelatively easy and requires relatively little computing resource usage.Preferably the two differences are compared with thresholds todistinguish motion from noise. That means that the processing means arearranged to establish a motion pattern by comparing the first differencewith a first predetermined motion threshold and the second differencewith a second predetermined motion threshold.

Typically, the first predetermined motion threshold and the secondpredetermined motion threshold are mutually equal. Optionally, thesecond value and the fourth value are mutually equal. Preferably, thesecond value is also based on a pixel value of another fields, e.g. thefirst one of the consecutive fields. Preferably, the fourth value isalso based on a pixel value of another field, e.g. the third one of theconsecutive fields.

In an embodiment of the motion sequence pattern detector according tothe invention the processing means are arranged to increase the value ofthe video motion measure if the first difference is larger than thefirst predetermined motion threshold and the second difference is largerthan the second predetermined motion threshold. In the case that themotion pattern comprises two relatively high values it is assumed thatthe motion pattern corresponds to video mode. As a consequence the valueof the video motion measure has to be increased.

In an embodiment of the motion sequence pattern detector according tothe invention the processing means are arranged to modify the value ofthe film motion measure if only the first difference is larger than thefirst predetermined motion threshold or only the second difference islarger than the second predetermined motion threshold. In the case thatthe motion pattern comprises one relatively high value and onerelatively low value it is assumed that the motion pattern correspondsto film mode. As a consequence the value of the film motion measure hasto be increased.

In an embodiment of the motion sequence pattern detector according tothe invention the processing means are arranged to establish a first oneof the motion patterns by:

computing a third difference between the first pixel value of the firstone of the consecutive fields and the third pixel value of the third oneof the consecutive fields;

computing a first minimum of the first difference and the thirddifference and assigning the first minimum to the first difference; and

computing a second minimum of the second difference and the thirddifference and assigning the second minimum to the second difference. Anadvantage of this embodiment is that it is arranged to correctly dealwith vertical detail, e.g. structures in the image which have a verticalsize substantially equal to the size of one video line. These structureswhich are present in e.g. the odd fields and not in the even fieldsmight be interpreted as motion. To overcome this misinterpretation thecomparison with the third difference is made.

An embodiment of the motion sequence pattern detector according to theinvention is arranged to output a signal indicating presence of filmmaterial at a location corresponding to a first one of the groups ofpixels on basis of comparing a first one of the motion patterns, withthe predetermined film motion pattern, the first one of the motionpatterns corresponding to the first one of the groups of pixels. Insteadof providing a classification value (film, video, hybrid or static) forthe field, more detailed information is provided, e.g. a kind of maskwhich represents which portions of the image correspond to film mode andwhich portions correspond to video mode.

An embodiment of the motion sequence pattern detector according to theinvention comprises a contrast measurement unit for selecting a firstone of the groups of pixels by means of:

computing a first value of a contrast measure for a first set of pixelsof the first one of the consecutive fields;

comparing the first value of the contrast measure with a predeterminedcontrast threshold; and

assigning the first set of pixels as the first one of the groups ofpixel if the first value of the contrast measure is higher than thepredetermined contrast threshold. By selecting pixels or groups ofpixels with a relatively high amount of contrast the noise sensitivityis reduced. In other words, an advantage of this motion sequence patterndetector is that it is more robust.

In an embodiment of the motion sequence pattern detector according tothe invention, the contrast measurement unit is arranged to compute thefirst value of the contrast measure on basis of calculating a firstdifference between the value of a first one of the pixels of the firstset of pixels and the value of another pixel of the first one of theconsecutive fields. This embodiment is arranged to compute spatialcontrast.

In an embodiment of the motion sequence pattern detector according tothe invention, the contrast measurement unit is arranged to compute thefirst value of the contrast measure on basis of calculating a seconddifference between the value of the first one of the pixels of the firstset of pixels and the value of a further pixel of a second one of theconsecutive fields. This embodiment is arranged to computespatio-temporal contrast.

An embodiment of the motion sequence pattern detector according to theinvention is arranged to compute a new predetermined contrast thresholdon basis of the number of times the values of the contrast measure beingcomputed for the first one of the consecutive fields have exceeded thepredetermined contrast threshold. In other words, the value of thecontrast threshold is dynamically adapted. As a consequence the numberof groups of pixels which is used for the motion pattern matching isrelatively constant over time. An advantage of this embodiment accordingto the invention is that the number of computations is relativelyconstant.

It is another object of the invention to provide an image processingapparatus of the kind described in the opening paragraph which comprisesa motion sequence pattern detector which is arranged to deal with hybridsequences and which is relatively simple.

This object of the invention is achieved in that the motion sequencepattern detector of the image processing apparatus, comprises processingmeans which is arranged:

to compute for a first one of the consecutive fields a value of a videomotion measure and a value of a film motion measure; and

to determine the presence of film material on basis of the value of thevideo motion measure and the value of the film motion measure,

the value of the video motion measure being computed by:

establishing a plurality of motion patterns for respective groups ofpixels of the first one of the consecutive fields;

comparing each of the plurality of motion patterns with a predeterminedvideo motion pattern and conditionally increasing the value of the videomotion measure, the value of the film motion measure being computed by:

comparing each of the plurality of motion patterns with a predeterminedfilm motion pattern and conditionally increasing the value of the filmmotion measure.

The image processing unit of the image processing apparatus mightsupport one or more of the following types of image processing:

Video compression, i.e. encoding or decoding, e.g. according to the MPEGstandard.

De-interlacing: Interlacing is the common video broadcast procedure fortransmitting the odd or even numbered image lines alternately.De-interlacing attempts to restore the full vertical resolution, i.e.make odd and even lines available simultaneously for each image;

Image rate conversion: From a series of original input images a largerseries of output images is calculated. Output images are temporallylocated between two original input images; and

Temporal noise reduction. This can also involve spatial processing,resulting in spatial-temporal noise reduction.

The image processing apparatus optionally comprises a display device fordisplaying the output images. The image processing apparatus optionallycomprises storage means for storage of images: either the input or theoutput images. The image processing apparatus might e.g. be a TV, a settop box, a VCR (Video Cassette Recorder) player, a satellite tuner, or aDVD (Digital Versatile Disk) player or recorder.

It is another object of the invention to provide a method of the kinddescribed in the opening paragraph which can deal with hybrid sequencesand which is relatively simple.

This object of the invention is achieved in that the method of detectingpresence of film material in a series of consecutive video fields,comprises:

computing for a first one of the consecutive fields a value of a videomotion measure and a value of a film motion measure; and

determining the presence of film material on basis of the value of thevideo motion measure and the value of the film motion measure, the valueof the video motion measure being computed by:

establishing a plurality of motion patterns for respective groups ofpixels of the first one of the consecutive fields;

comparing each of the plurality of motion patterns with a predeterminedvideo motion pattern and conditionally increasing the value of the videomotion measure, the value of the film motion measure being computed by:

comparing each of the plurality of motion patterns with a predeterminedfilm motion pattern and conditionally increasing the value of the filmmotion measure.

It is another object of the invention to provide a computer programproduct of the kind described in the opening paragraph which can dealwith hybrid sequences and which is relatively simple.

This object of the invention is achieved in that the computer programproduct after being loaded, providing said processing means with thecapability to carry out the following steps:

computing for a first one of the consecutive fields a value of a videomotion measure and a value of a film motion measure; and

determining the presence of film material on basis of the value of thevideo motion measure and the value of the film motion measure, the valueof the video motion measure being computed by:

establishing a plurality of motion patterns for respective groups ofpixels of the first one of the consecutive fields;

comparing each of the plurality of motion patterns with a predeterminedvideo motion pattern and conditionally increasing the value of the videomotion measure, the value of the film motion measure being computed by:

comparing each of the plurality of motion patterns with a predeterminedfilm motion pattern and conditionally increasing the value of the filmmotion measure. Modifications of the motion sequence pattern detectorand variations thereof may correspond to modifications and variationsthereof of the method, of the computer program product and of the imageprocessing apparatus described.

These and other aspects of the motion sequence pattern detector, of themethod, of the computer program product and of the image processingapparatus according to the invention will become apparent from and willbe elucidated with respect to the implementations and embodimentsdescribed hereinafter and with reference to the accompanying drawings,wherein:

FIG. 1A schematically shows two fields of one frame;

FIG. 1B schematically shows 2:2 pull-down;

FIG. 1C schematically shows 3:2 pull-down;

FIG. 2 schematically shows three consecutive video fields;

FIG. 3A schematically shows an embodiment of the motion sequence patterndetector according to the invention;

FIG. 3B schematically shows an embodiment of the motion sequence patterndetector according to the invention, comprising a contrast measurementunit;

FIG. 4 schematically shows a two-dimensional feature space;

FIG. 5 schematically shows a two-dimensional mask indicating the type ofmode; and

FIG. 6 schematically shows an embodiment of the image processingapparatus according to the invention.

Same reference numerals are used to denote similar parts throughout thefigs.

FIG. 1A schematically shows two successive fields 100, 102 of a videosignal. The first field 100 comprises the pixel values, e.g. 104-112 ofthe odd lines of the frame and the second field 102 comprises the pixelvalues, e.g. 114-122 of the even lines of the frame. For instance atframe coordinates corresponding to pixel 116 of the second field 102there is no pixel value 124 directly available in the first field 100.That means that if a pixel value 124 is required that this pixel valuehas to be derived from other pixel values. For example, this pixel valueis derived, i.e. can be calculated by means of an interpolation of pixelvalues of the first field 100, e.g. by means of an interpolation basedon the pixel values 104-109. Optionally less pixel values are taken intoaccount. An interpolation might also include an order statisticaloperation such as a median operation. It may also include pixels fromfield 102 or from a (not depicted) field preceding field 100.

FIG. 1B schematically shows 2:2 pull-down. An input stream of pictures130-136 with a frequency of 25 Hz is up-converted to an output stream ofvideo fields 138-152 with a frequency of 50 Hz. The different phases{0,1} of the video fields are denoted below the video fields 138-152.This film phase indicates the position in the repetition pattern and istypically calculated in a film detector.

FIG. 1C schematically shows 3:2 pull-down. An input stream of pictures160-164 with a frequency of 24 Hz is up-converted to an output stream ofvideo fields 168-182 with a frequency of 60 Hz. The different phases{0,1,2,3,4} of the video fields are denoted below the video fields168-182.

FIG. 2 schematically shows a number of pixels 202-222 of threeconsecutive video fields: current c, previous p and pre-previous pp. Thecurrent field corresponds with n, the previous field corresponds withn−1 and the pre-previous corresponds with n−2. The current field c andthe pre-previous field pp comprise even lines and the previous field pcomprises odd lines. In this document, a pixel value of a pixel isdenoted with a three-dimensional luminance function F({right arrow over(x)},n), with the vector {right arrow over (x)} comprising two spatialcoordinates x and y. The pixels 202-208 of the pre-previous field ppcorrespond to pixels of a column with a certain x-coordinate which isequal to the x-coordinate of the column to which the pixels 210-214 ofthe previous field p belong and equal to the x-coordinate of the columnto which the pixels 216-222 of the current field c belong. For some ofthe pixels the coordinates are depicted. E.g. pixel 204 has coordinates(x,y,n−2) and pixel 210 has coordinates (x,y−1,n−1).

As explained in connection with FIG. 1A it is possible to determinepixel values for pixels for which there is no pixel value directlyavailable. E.g. the value for a pixel with coordinates (x,y,n−1) mightbe determined by means of pixel values in the spatio-temporalenvironment of (x,y,n−1).

FIG. 3A schematically shows an embodiment of the motion sequence patterndetector 300 according to the invention, comprising:

a number of input connections for providing the motion sequence patterndetector 300 with luminance values of respective pixels;

a number of de-interlacing units 302 and 304;

a number of subtraction units 306-310 for calculating the absolutedifference between two incoming values;

a number of minimum operators 312 and 314 for determining the minimum oftwo incoming values;

a number of comparators 316 and 318 for detecting whether an incomingvalue is higher than a predetermined threshold;

a logical unit 320 comprising a number of inverters and and-operators;

a number of counters 322-326;

a combining unit 328 for combining the results of the counters 322-326;

a number of output connectors 330 and 332;

a control interface 334 for resetting the values of the counters 322-326after the computations for a field have been completed; and

a number of control interface 336 and 338 for adapting the values of thefirst predetermined motion threshold T_(m) ^(p) and the secondpredetermined motion threshold T_(m) ^(c). The motion sequence patterndetector 300 may be implemented using one processor. Normally, thesefunctions are performed under control of a software program product.During execution, normally the software program product is loaded into amemory, like a RAM, and executed from there. The program may be loadedfrom a background memory, like a ROM, hard disk, or magnetically and/oroptical storage, or may be loaded via a network like Internet.Optionally an application specific integrated circuit provides thedisclosed functionality.

The working of the motion sequence pattern detector is as follows.

Suppose that for a particular pixel 218 with coordinates (x,y,n) themode has to be determined. The motion sequence pattern detector 300 isprovided with a number of pixel values. Alternatively the motionsequence pattern detector 300 is arranged to access a memory device 342to retrieve these pixel values. This embodiment requires the followingpixel values F(x,y,n),F(x,y,n−2), F(x,y−1,n−1) and F(x,y+1,n−1) in orderto determine the mode for pixel 218 with coordinates (x,y,n). (See alsoFIG. 2)

On basis of three of these pixel values a first estimate {tilde over(F)}₁(x,y,n−1) is computed for the pixel with coordinates (x,y,n−1).This is done by the first de-interlacing unit 304. In this case thede-interlacing is based on a median operation as specified in Equation1.{tilde over (F)}₁(x,y,n−1)=Median(F(x,y−1,n−1), F(x,y+1,n−1),F(x,y,n−2))  (1)

Alternatively other types of de-interlacing can be applied, e.g. onbasis of an averaging operation.

On basis of three of the input pixel values also a second estimate{tilde over (F)}₂(x,y,n−1) is computed for the pixel with coordinates(x,y,n−1). This is done by the second de-interlacing unit 302. In thiscase the de-interlacing is based on a median operation as specified inEquation 2:{tilde over (F)}₂(x,y,n−1)=Median(F(x,y−1,n−1), F(x,y+1,n−1), F(x,y,n))  (2)

The next step comprises computing:

a first difference δ_(p)(x, y) between a first pixel value F(x,y,n−2) ofthe first one of the consecutive fields pp and the first estimate {tildeover (F)}₁(x,y,n−1) being derived from a second one of the consecutivefields p, as specified in Equation 3; and

a second difference δ_(c)(x,y) between a third pixel value F(x,y,n) of athird one of the consecutive fields c and the second estimate {tildeover (F)}₁(x,y,n−1) being derived from the second one of the consecutivefields p, as specified in Equation 4.δ_(p)(x,y)=|F(x,y,n−2)−{tilde over (F)} ₁(x,y,n−1)|  (3)δ_(c)(x,y)=|F(x,y,n)−{tilde over (F)} ₂(x,y,n−1)|  (4)

The next step comprises:

computing a third difference δ_(f)(x,y) between the first pixel valueF(x,y,n−2) of the first one of the consecutive fields pp and the thirdpixel value F(x,y,n) of the third one of the consecutive fields c, asspecified in Equation 5;

computing a first minimum δ_(p)′(x,y) of the first difference δ_(p)(x,y)and the third difference δ_(f)(x,y) and assigning the first minimum tothe first difference, as specified in Equation 6; and

computing a second minimum δ_(c)′(x,y) of the second differenceδ_(c)(x,y) and the third difference δ_(f)(x,y) and assigning the secondminimum to the second difference, as specified in Equation 7.δ_(f)(x,y)=|F(x,y,n−2)−F(x,y,n)|  (5)δ_(p)′(x,y)=min(δ_(p)(x,y), δ_(f)(x,y))   (6)δ_(c)′(x,y)=min(δ_(c)(x,y),δ_(f)(x,y))   (7)

The next step comprises comparing the first difference δ_(p)′(x,y) witha first predetermined motion threshold T_(m) ^(p) and the seconddifference δ_(c)′(x,y) with a second predetermined motion thresholdT_(m) ^(c). This is done by means of comparators 318 and 316,respectively. The comparator 318 provides Boolean values M_(p)(x,y) asoutput, which indicate whether there is movement between the firstderived pixel with coordinates (x,y,n−1) and the pixel 204 withcoordinates (x,y,n−2). The comparator 316 provides Boolean valuesM_(c)(x,y) as output, which indicate whether there is movement betweenthe particular pixel 218 with coordinates (x,y,n) and the second derivedpixel with coordinates (x,y,n−1). The input-output relation ofcomparator 318 is specified in Equation 8 and the input-output relationof comparator 316 is specified in Equation 9:If δ_(p)′(x,y)>T _(m) ^(m) then M _(p)(x,y)=1 else M _(p)(x,y)=0   (8)If δ_(c)′(x,y)>T _(m) ^(c) then M _(c)(x,y)=1 else M _(c)(x,y)=0   (9)

Table 1 shows the four different possible combinations of the values ofM_(c)(x, y) and M_(p)(x, y). These combinations correspond to possiblemotion patterns 1-4. For each of these patterns Table 1 indicateswhether the motion pattern is a predetermined video motion pattern orone of the predetermined film motion patterns. TABLE 1 Motion patternsPattern identification M_(p)(x, y) M_(c)(x, y) Type of motion pattern 10 0 No movement, type unknown 2 0 1 Film motion pattern, phase A 3 1 0Film motion pattern, phase B 4 1 1 Video motion pattern

Hence, on basis of the values of M_(c)(x,y) and M_(p)(x,y) the mode forthe particular pixel 218 is determined.

The mode is determined for a large number N of pixels of each field,e.g. for 25% of the pixels of a field. The pixels might be selected onbasis of a simple sub-sampling strategy. The results of the modedeterminations are accumulated by means of a number of counters 322-326.Each time a pattern with identification 2 is detected then the value ofS_(film) ^(A) is increased with 1, as specified in Equation 10:$\begin{matrix}{S_{film}^{A} = {\sum\limits_{N}\left\{ {1\left. {{M_{p}\left( {x,y} \right)} = {{0\bigwedge{M_{c}\left( {x,y} \right)}} = 1}} \right\}} \right.}} & (10)\end{matrix}$Each time a pattern with identification 3 is detected then the value ofS_(film) ^(B) is increased with 1, as specified in Equation 11:$\begin{matrix}{S_{film}^{B} = {\sum\limits_{N}\left\{ {1\left. {{M_{p}\left( {x,y} \right)} = {{1\bigwedge{M_{c}\left( {x,y} \right)}} = 0}} \right\}} \right.}} & (11)\end{matrix}$Each time a pattern with identification 4 is detected then the value ofS_(video) is increased with 1, as specified in Equation 12:$\begin{matrix}{S_{video} = {\sum\limits_{N}\left\{ {1\left. {{M_{p}\left( {x,y} \right)} = {{1\bigwedge{M_{c}\left( {x,y} \right)}} = 1}} \right\}} \right.}} & (12)\end{matrix}$

Eventually the values S_(film) ^(A), S_(video) and S_(film) ^(B), of thecounters 322-326 are combined by means of combining unit 328. One of theoperations being performed by the combining unit 328 is specified inEquation 13. The reason for the subtraction of the “min”-term is toeliminate the effect of covering and uncovering. This subtraction isoptionally.S _(film) =|S _(film) ^(A) −S _(film) ^(B)|−min(S _(film) ^(A) , S_(film) ^(B))   (13)Finally a vector S comprising two values is achieved as denoted inEquation 14:{right arrow over (S)}=(S _(film,) S _(video))   (14)This vector {right arrow over (S)} can be used to detect the mode usinga set of thresholds as depicted in FIG. 4. The mode is provided at theoutput connector 330. Optionally, the vector {right arrow over (S)} isprovided at the output connector 330. Optionally a two-dimensional maskindicating the type of mode per pixel or group of pixels is provided atthe output connector 332. (See FIG. 5)

FIG. 3B schematically shows an embodiment of the motion sequence patterndetector 301 according to the invention, comprising a contrastmeasurement unit 340. The contrast measurement unit 340 is arranged tomake a selection of groups of pixels on basis of the pixel values of thevideo fields. More particular on basis of differences between pixelvalues.

Suppose that each of the groups of pixels contain one respective pixel.Deciding whether a particular pixel is to be selected for the motionpattern detection, comprises the following steps:

computing the value of a contrast measure C¹(x,y,n) for the particularpixel;

comparing the value of the contrast measure C¹(x,y,n) with apredetermined contrast threshold T_(c)(n); and

assigning the particular pixel as the first one of the groups of pixelif the value of the contrast measure C¹(x,y,n) is higher than thepredetermined contrast threshold T_(c)(n).

By testing a large number of pixels of a video field with coordinate n acollection B(n) of groups of pixels is created for that field. Thecollection B(n) is specified by means of Equation 15:B(n)={(x,y)|∀C ¹(x,y,n)>T _(c)(n)}  (15)

For calculating a contrast measure C¹(x,y,n) spatial or temporal pixels,related to (x,y,n), can be applied. Optionally, multiple comparisons aremade. This will be explained by means of some examples.

Suppose that the value of a first contrast measure C¹(x,y,n) is computedon basis of calculating a first difference between the value of theparticular pixel and the value of another pixel of the same field, asspecified if Equation 16:C ¹(x,y,n)=F(x,y,n)−F(x,y−2,n)   (16)

Suppose that the value of a second contrast measure C²(x,y,n) iscomputed on basis of calculating a second difference between the valueof the particular pixel and the value of a further pixel of the samefield, as specified if Equation 17:C ²(x,y,n)=F(x,y,n)−F(x−1,y,n)   (17)

Suppose that the value of a third contrast measure C³(x,y,n) is computedon basis of calculating a third difference between the value of theparticular pixel and the value of a pixel of the another field, asspecified if Equation 18:C ³(x,y,n)=F(x,y,n)−F(x,y,n−2)   (18)

Equation 15 can be rewritten into Equation 19:B(n)={(x,y)|∀(C ¹(x,y,n)>T _(c)(n)ˆC ²(x,y,n)>T _(c)(n)ˆC ³(x,y,n)>T_(c)(n))}  (19)

It will be clear that alternative approaches can be applied to estimatelocal contrast, i.e. to calculate a contrast measure C¹(x,y,n). Onlythose pixels which have a relatively high contrast compared to theirspatio-temporal environment are selected for the motion patterndetection.

Preferably the value of the contrast threshold T_(c)(n) is dynamicallyadapted. E.g. if the actual selected groups of pixels for a particularfield is higher than a target value, then the value of the contrastthreshold T_(c)(n+1) for the next field is based on an increased valueof T_(c)(n). If the actual selected groups of pixels for a particularfield is lower than a target value, then the value of the contrastthreshold T_(c)(n+1) for the next field is based on a decreased value ofT_(c)(n). The target value might be equal to 20% of the total number ofpixels of the field. As a consequence the number of groups of pixelsbeing used per field for the motion pattern matching is relativelyconstant over time. An advantage of this embodiment according to theinvention is that the number of computations is relatively constant.

Optionally the values of the first predetermined motion threshold T_(m)^(p) and the second predetermined motion threshold T_(m) ^(c) depend on,the value of the contrast threshold T_(c)(n), e.g. as specified inEquations 20 and 21:T _(m) ^(p)(n)=0.5T _(c)(n)   (20)T _(m) ^(c)(n)=0.5T _(c)(n)   (21)This means that the motion thresholds are high for fields with highcontrast; so the motion sequence pattern detector becomes relativelyinsensitive to noise without loss of motion sensitivity. So, anadvantage of this embodiment is graceful degradation, since the tradeoff between noise sensitivity and motion sensitivity is automaticallyadapted to the contrast in the video signal.

FIG. 4 schematically shows a two-dimensional feature space. The x-axis402 corresponds with the parameter S_(film) as specified in Equation 13.The y-axis 404 corresponds with the parameter S_(video) as specified inEquation 12. Note that the two axes are normalized to the total numberof pixels used to classify the motion pattern. That means that alocation in the two-dimensional feature space corresponds with thevector {right arrow over (S)}=(S_(film), S_(video)). The two-dimensionalfeature space is divided into a number of regions by means of a numberof boundaries 406-410. Each of the regions corresponds with a certainmode. In other words, based on the computed {right arrow over(S)}=(S_(film), S_(video)) and the rules for classification asschematically provided by means of FIG. 4 the eventual mode for aparticular field can be determined:

I: The field primarily comprises material originating from an interlacedvideo camera and hence the field corresponds to video mode;

II: The field primarily comprises material originating from a filmcamera and hence the field corresponds to film mode;

III: The field comprises material originating from an interlaced videocamera but also material originating from a film camera and hence thefield corresponds to a hybrid mode;

IV: No significant motion has been detected and hence the fieldcorresponds to a static mode.

FIG. 5 schematically shows a two-dimensional mask 500 indicating thetypes of mode of a field of a hybrid sequence. Most of the field 504comprises material which originates from a film camera and only arelatively small portion 502 corresponds to video material. A mask asdepicted in FIG. 5 is an output of the motion sequence pattern detector300 and is provided at the output connector 332.

FIG. 6 schematically shows an embodiment of the image processingapparatus 600 according to the invention, comprising:

Receiving means 602 for receiving a signal representing input imagescomprising video fields. The signal may be a broadcast signal receivedvia an antenna or cable but may also be a signal from a storage devicelike a VCR (Video Cassette Recorder) or Digital Versatile Disk (DVD).The signal is provided at the input connector 610;

The motion sequence pattern detector 608 as described in connection withany of the FIGS. 3A or 3B;

An image processing unit 604 for calculating a sequence of output imageson basis of the succession of video fields. The image processing unit604 is controlled by the motion sequence pattern detector 608. Controlmeans that the output of the motion sequence pattern detector 608influences the image processing unit 604. For instance, if the imageprocessing unit 604 is arranged to perform de-interlacing then theoutput (mode and phase) is used to combine corresponding video fields toimages; and

A display device 606 for displaying the output images of the imageprocessing unit 604. This display device 606 is optional.

The image processing apparatus 600 might e.g. be a TV. Alternatively theimage processing apparatus 600 does not comprise the optional displaydevice 606 but provides the output images to an apparatus that doescomprise a display device 606. Then the image processing apparatus 600might be e.g. a set top box, a satellite-tuner, a VCR player, a DVDplayer or a DVD recorder. Optionally the image processing apparatus 600comprises storage means, like a hard-disk or means for storage onremovable media, e.g. optical disks. The image processing apparatus 600might also be a system being applied by a film-studio or broadcaster.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention and that those skilled in the art willbe able to design alternative embodiments without departing from thescope of the appended claims. In the claims, any reference signs placedbetween parentheses shall not be constructed as limiting the claim. Theword ‘comprising’ does not exclude the presence of elements or steps notlisted in a claim. The word “a” or “an” preceding an element does notexclude the presence of a plurality of such elements. The invention canbe implemented by means of hardware comprising several distinct elementsand by means of a suitable programmed computer. In the unit claimsenumerating several means, several of these means can be embodied by oneand the same item of hardware.

1. A motion sequence pattern detector for detecting presence of filmmaterial in a series of consecutive video fields, the motion sequencepattern detector comprising processing means which is arranged: tocompute for a first one of the consecutive fields a value of a videomotion measure and a value of a film motion measure; and to determinethe presence of film material on basis of the value of the video motionmeasure and the value of the film motion measure, the value of the videomotion measure being computed by: establishing a plurality of motionpatterns for respective groups of pixels of the first one of theconsecutive fields; comparing each of the plurality of motion patternswith a predetermined video motion pattern and conditionally increasingthe value of the video motion measure, the value of the film motionmeasure being computed by: comparing each of the plurality of motionpatterns with a predetermined film motion pattern and conditionallyincreasing the value of the film motion measure.
 2. A motion sequencepattern detector as claimed in claim 1, wherein the groups of pixelseach have one pixel.
 3. A motion sequence pattern detector as claimed inclaim 1, wherein the processing means are arranged to establish a firstone of the motion patterns by computing: a first difference between afirst pixel value of the first one of the consecutive fields and asecond value being derived from a second one of the consecutive fields;and a second difference between a third pixel value of a third one ofthe consecutive fields and a fourth value being derived from the secondone of the consecutive fields.
 4. A motion sequence pattern detector asclaimed in claim 3, wherein the processing means are arranged toestablish the first one of the motion patterns by comparing the firstdifference with a first predetermined motion threshold and the seconddifference with a second predetermined motion threshold.
 5. A motionsequence pattern detector as claimed in claim 4, wherein the processingmeans are arranged to establish a first one of the motion patterns by:computing a third difference between the first pixel value of the firstone of the consecutive fields and the third pixel value of the third oneof the consecutive fields; computing a first minimum of the firstdifference and the third difference and assigning the first minimum tothe first difference; and computing a second minimum of the seconddifference and the third difference and assigning the second minimum tothe second difference.
 6. A motion sequence pattern detector as claimedin claim 4, wherein the processing means are arranged to increase thevalue of the video motion measure if the first difference is larger thanthe first predetermined motion threshold and the second difference islarger than the second predetermined motion threshold.
 7. A motionsequence pattern detector as claimed in claim 4, wherein the processingmeans are arranged to modify the value of the film motion measure ifonly the first difference is larger than the first predetermined motionthreshold or only the second difference is larger than the secondpredetermined motion threshold.
 8. A motion sequence pattern detector asclaimed in claim 1, being arranged to output a signal indicatingpresence of film material at a location corresponding to a first one ofthe groups of pixels on basis of comparing a first one of the motionpatterns, with the predetermined film motion pattern, the first one ofthe motion patterns corresponding to the first one of the groups ofpixels.
 9. A motion sequence pattern detector as claimed in claim 1,comprising a contrast measurement unit for selecting a first one of thegroups of pixels by means of: computing a first value of a contrastmeasure for a first set of pixels of the first one of the consecutivefields; comparing the first value of the contrast measure with apredetermined contrast threshold; and assigning the first set of pixelsas the first one of the groups of pixel if the first value of thecontrast measure is higher than the predetermined contrast threshold.10. A motion sequence pattern detector as claimed in claim 9, whereinthe contrast measurement unit is arranged to compute the first value ofthe contrast measure on basis of calculating a first difference betweenthe value of a first one of the pixels of the first set of pixels andthe value of another pixel of the first one of the consecutive fields.11. A motion sequence pattern detector as claimed in claim 10, whereinthe contrast measurement unit is arranged to compute the first value ofthe contrast measure on basis of calculating a second difference betweenthe value of the first one of the pixels of the first set of pixels andthe value of a further pixel of a second one of the consecutive fields.12. A motion sequence pattern detector as claimed in claim 9, which isarranged to compute a new predetermined contrast threshold on basis ofthe number of times the values of the contrast measure being computedfor the first one of the consecutive fields have exceeded thepredetermined contrast threshold.
 13. An image processing apparatus,comprising: receiving means for receiving a signal corresponding to aseries of consecutive video fields ; a motion sequence pattern detectoras claimed in claim 1; and an image processing unit for computing asequence of output images on basis of the series of consecutive videofields the image processing unit being controlled by the motion sequencepattern detector.
 14. An image processing apparatus as claimed in claim13, characterized in further comprising a display device for displayingthe output images.
 15. An image processing apparatus as claimed in claim14, characterized in that it is a TV.
 16. An image processing apparatusas claimed in claim 13, characterized in further comprising storagemeans for storage of the output images.
 17. An image processingapparatus as claimed in claim 16, characterized in that it is a DVDrecorder.
 18. A method of detecting presence of film material in aseries of consecutive video fields, comprising: computing for a firstone of the consecutive fields a value of a video motion measure and avalue of a film motion measure; and determining the presence of filmmaterial on basis of the value of the video motion measure and the valueof the film motion measure, the value of the video motion measure beingcomputed by: establishing a plurality of motion patterns for respectivegroups of pixels of the first one of the consecutive fields; comparingeach of the plurality of motion patterns with a predetermined videomotion pattern and conditionally increasing the value of the videomotion measure, the value of the film motion measure being computed by:comparing each of the plurality of motion patterns with a predeterminedfilm motion pattern and conditionally increasing the value of the filmmotion measure.
 19. A computer program product to be loaded by acomputer arrangement, comprising instructions to detect presence of filmmaterial in a series of consecutive video fields, the arrangementcomprising processing means and a memory, the computer program product,after being loaded, providing said processing means with the capabilityto carry out the following steps: computing for a first one of theconsecutive fields a value of a video motion measure and a value of afilm motion measure; and determining the presence of film material onbasis of the value of the video motion measure and the value of the filmmotion measure, the value of the video motion measure being computed by:establishing a plurality of motion patterns for respective groups ofpixels of the first one of the consecutive fields; comparing each of theplurality of motion patterns with a predetermined video motion patternand conditionally increasing the value of the video motion measure, thevalue of the film motion measure being computed by: comparing each ofthe plurality of motion patterns with a predetermined film motionpattern and conditionally increasing the value of the film motionmeasure.